Generalizing Criterion-related Validity Evidence for Certification Requirements Across Situations and Specialty Areas

DOCUMENT RESUME TM 017 260 Kane, Michael T. Generalizing Criterion-Related Validity Evidence for Certification Requirements across Situations and Specialty Areas. American Coll. Testing Program, Iowa City, Iowa. ACT-RR-90-3 May 90 26p. ACT Research %eport Series, P.O. Box 168, Iowa City, IA 52243. Reports Evaluative/Feasibility (142) MF01/PCO2 Plus Postage. *Certification; Comparative Analysis; *Concurrent Validity; *Criterion Referenced Tests; Ge.teralization; *Licensing Examinations (Professions); *Meta Analysis; Models; Occupational Tests; *Test Validity *Validity Generalization Developing good criterion measures of professional performance is difficult. If criterion-related validity evidence for certificaticn requirements could not be generalized beyond the specific context in which it was obtained, gathering that evidence would probably not :oe worth the effort. This paper examines two possible approaches to the generalization of criterion-related evidence for certification requirements. The first, validity generalization (meta-analysis), provides a statistical technique for generalizing tr,: results of particular studies. However, the criterion problem remaimi; generalizing fluff (criterion-related evidence based on weak or inappropriate criteria) merely results in a more general kind of fluff. The second approach uses substantive models as the basis for generalizing validity data; this approach offers several advantages, including more emphasis on the nature of the c:iterion and, possibly, some help in developing better criteria. Four reasons for usLrIg substantive models in stead of statistical meta-analysis models are discussed, and four major conclusions are considered. A 27-item list of references is included. (Author/SLD) *********************************************************************** Reproductions supplied by E^RS are the best that can be made from the original document. *********************************************************************** n . ACT Research Report Series 90-3 Generalizing Criterion-related Validity Evidence for Certification Requirements Across Situations and Specialty Areas